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Introduction 

The goal of the present note is twofold. Firstly, we correct some points in the 
paper (quoted in the following as [P]): 

P. Pragacz: "A generalization of the Macdonald-You formula" , Journal 
of Algebra 204, 573-587 (1998). 

Secondly, by comparing the formula appearing in the title of [P] with results of 
Stembridge [St] and some other combinatorial results, we deduce some new iden- 
' tities in Propositions 2-6. They concern: restrictions of Schubert classes to the 
ON ■ cohomology of Lagrangian Grassmannians as well as relations between Q-functions, 
Stembridge's coefficients, and various "hook numbers" . Also, we provide some ex- 
amples illustrating [P] and the formulas given in the present note. 

In this note, any unexplained notation or quotation stems from [P]. However, in 
order to make the notation maximally compatible with that used in [St] (which is 
our prinicipal reference here), we label strict partitions by A, and \i usually denotes 
an ordinary partition, contrariwise to [P]. 



1. Erratum to [P] 

Due to some bugs in the computer system SCHUR [Sch], [P, Example 3(b)] was 
miscalculated: the quadratic expression in Q-functions displayed there, written as 
a Z-linear combination of Q-functions, contains no negative summands. Conse- 
quently the sentence on p. 585, lines 6-7 from the bottom, is to be withdrawn from 
[P]. (These corrections do not affect other results of [P], in particular the main 
formulas.) 

2. Nonnegativity of the restriction coefficients 

In fact, if 

(!) **(>^) = X! CA ^' 

A 

with ca m G Z, then all the coefficients are nonnegative. Perhaps the easiest way 
to see this, is the following. Let for a G H*(G; Z), J G a stand for the degree of the 



This note was supported by KBN grant No. 2P03A 05112. 



2 



PIOTR PRAGACZ 



top codimensional component of a, and define similarly f G ,b for b G H*(G';Z). 
Given a strict partition A C (n, n — 1, . . . , 1), we denote by A v the strict partition 
whose parts complement those of A in {1, . . . , n}. We record the following property 
[P2]: 

Lemma 1 (Duality). The basis {o~' x } of the group H 2p (G';Z) and the basis 
{o~' X v} of the group H n ( n+1 )~ 2p (G'; Z) are dual under the pairing (a, b) i— > J G , a ■ b 
of Poincare duality. 

Now, if i*(o~^) = X]a c V°A' w hh c\^ G Z, then it follows from the duality 
property that 

(2) c AAt = / r((7 M ) -a' xv . 

JG' 



Using the projection formula for i, this is rewritten as 




Regard G as a homogeneous space GL(V) / P, where P is a suitable parabolic 
subgroup of GL(V). Let O C G be a Schubert variety representing a M and let 
O' C G" C G be a Schubert variety representing cx A v • Using e.g. Kleiman's theorem 
on a general translate [K], we can replace O by a translate by an element g G GL(V) 
such that g ■ O and O' meet properly, and this intersection is represented as a 
nonnegative zero-cycle. This shows that ca m > 0. 

A similar property holds in the following more general setting. Let now G D 
P D B be a semisimple linear algebraic group, a parabolic subgroup, and a Borel 
subgroup. In a generalized flag variety G/P, one has Schubert varieties BwP/P 
and their Schubert classes in if*(G/P;Z) indexed by a corresponding subset of 
the Weyl group. These Schubert classes enjoy a similar duality property. In an 
analogous way, using a general translate argument, one shows that the fundamental 
class of any subscheme of G/P is a Z-linear combination of the Schubert classes in 
H*(G/P; Z) with nonnegative coefficients. Combining this with a well-known fact 
about pulling back the class of a Cohen-Macaulay subscheme (see, e.g., Lemma on 
p. 108 in [F-P]), we get the following result (also implying the nonnegativity of the 
above c A ^): 

Proposition 1. Let f : G/P — > Y be morphism to a nonsingular variety Y . Let 
Z be a pure- dimensional closed Cohen-Macaulay subscheme of Y . Then f*([Z]) 
is a Z-linear combination of the Schubert classes in H*(G/P;Z) with nonnegative 
coefficients. 

3. Stembridge's coefficients 

We recall (see the discussion after [P, Proposition 8]) that the coefficients ap- 
pearing in (1) and those appearing in: 



(4) 
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satisfy = g\^. Here, we take suficiently large Grassmannians % : G' <^-> G. 
To be more precise, this means that given we take n > \fi\ so that any strict 
partition A with |A| = is contained in (n,n — 1, . . . , 1). Consequently, all the 
coefficients are nonnegative. But this result, together with a combinatorial 
interpretation of the g\^s, was already established by Stembridge in [St]. 1 Indeed, 
the last displayed (unnumbered) equality before [St, Theorem 9.3]: 

(5) "S M = J2 9x»Qx" 

\eDP n 

is identical with (4) because in the notation of [St] (and [M]) is equal to ??(s M ) 
in our notation. 2 In [St], (5) is a consequence of the equality 

(6) P\ = 

H=|A| 

where P\ = 2~ l ^Q\, and comparison of the canonical scalar products on the ring of 
all symmetric functions with that on the ring of C}-functions. To the nonnegativity 
of is given, in loc. cit., several interpretations in representation theory, some of 
which go back to Morris and Stanley. 

(Observe that (4) and (6) yield the following expression for r}(P\): 

(7) V(P\) = Yl 9\M S ^) = Yl Y 9\»9vnQv) 

H = |A| | M | = |A|M = |A| 

Stembridge [St] also established a combinatorial interpretation of the numbers 
f^u appearing as coefficients in the expansion: 

(8) P^ = Yf^ 

A 

where fx, z/, and A denote now strict partitions. It will be convenient to set 

(9) e* v := 2^)+*H-*(A) ^ 

There exists a geometric analogue of (8): in the cohomology ring H*(G';Z) of a 
sufficiently large Lagrangian Grassmannian, 

(10) <-< = E e >A- 

A 

(See [P2, Sect.6].) 

Stembridge's combinatorial description of the above and gx^ can be summa- 
rized by the following: 



1 The fact that this result was already established by Stembridge, has been learned by the 
author only in June 1999. 
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Theorem [St], (%) The coefficient is equal to the number of marked shifted 
tableaux T of shape A//i and weight (or content) v such that: 

(a) The word w(T) associated with T ([St, Sect. 8] and [M, p. 258]) has the lattice 
property in the sense of loc.cit.; 

(b) for each k > 1, the rightmost occurence of k' in w(T) precedes the last occurence 
ofk. 

(ii) The coefficient g\^ is equal to the number of unshifted marked tableaux T of 
shape \i and weight A satisfying (a) and (b) above. 

For all unexplained here combinatorial notions, we refer the reader to [St, Sect. 6 
and 8], [P2, Sect. 4], and to [M, III. 8 pp. 255-259]. We make no attempt to make a 
complete survey here. Some examples of the coefficients g\^ will be given below. 

Summarizing the content of this section, we record: 
Proposition 2. We have for a partition \i C (n n ) 

(11) *>m)=E^ <7 A. 

A 

where A runs over strict partitions contained in (n, n — 1,. . . , 1), and g\^ is the 
Stembridge coefficient described in Theorem (ii). 

4. Quadratic relations between Q-functions 

We pass now to some applications of the generalized Macdonald-You formula 
([L-L2], [P, Corollary 2]): 

(12) ^(Sm) = ^2Q(a il ,...,a ik ) ■ QA#Bx(a il ,...,« Sfc )- 

Recall that here, for [i = (cki, ... , a n \Pi, . . . , (3 n ) in Frobenius notation, 

(13) A = (ai,... ,a n ) := (ai + l,... ,a n + l), £:= (ft, ...,/?„) , 

and the sum is over all sequences 1 < %\ < ■ ■ ■ < ik < n and k = 0, 1, . . . , n. 

Since 77 (e^) = t](hi), where hi is the zth complete homogeneous symmetric func- 
tion, we have for a partition fx 

(14) V(s^) = v(s^), 

where /U~ = (ft,... , f3 n \ai, . . . , a n ) is the conjugate partition of fx. We set in 
addition 

(15) C = (ci, . . . ,c n ) := (ft + 1, . . . ,ft + 1) , D :=(«!,... ,a n ). 

rni /n\ „„j /1 a\ j-U„ -C„n 
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Proposition 3. We have 

(16) £Q( o il ,...,a ifc ) ' QAp\(a il ,...,a,J — ^ < 3(c il ,c ifc ) ' < 2c#D\(c; 1 ,c ife ) 7 

where the sums are over all sequences 1 < %\ < • • ■ < < n and k = 0, 1, . . . , n. 

The relations (16), regarded from the side of C}-functions, seem to be rather 
nontrivial. For instance, for fi = (5 3 31 3 ) = (432|621), so A = (5,4,3), B = (6,2, 1), 
C = (7, 3, 2), and D = (4, 3, 2), we get the equation 

Q654321 — Q$ ■ £?64321 + Qa ' £?65321 — Q3 ' ^65421 — £?54 ' Q6321 

(17) + Q53 " Q6421 - Q43 ' Q6521 + Q543 ' ^621 
= Q32 ■ Q7432 + Q732 ■ Qa32- 

Using a (hopefully) debugged version of SCHUR, (16) is expressed as the following 
Z-linear combination of the Q\S : 

8<5n 64 + 8Qn 631 + 8Qll 541 + 8Q11 532 + 8QlO 74 

+ 8Q10 731 + 8Q10 65 + 24C}io 641 + 24C}io 632 + 24C}io 542 

+ 8Q10 5321 + 8Q975 + I6Q974I + I6Q9732 + 16C}g651 

+ 48C}9642 + 16C}g6321 + I6Q9543 + I6Q9542I + 8Q8751 

+ 24Q 87 42 + 8Q 873 21 + 24Q 8652 + 24Q 86 43 + 24^86421 

+ 8Q85431 + 8C}7653 + 8Q76521 + 8Q7643I 

Consequently, taking sufficiently large Grassmannians i : G' 67, we have 

«*(o"5 3 31 3 ) = 



_/ 

°11 64 


+ °11 631 


+ °11 541 


+ °11 532 


+ cr' w 74 


+ a 10 731 


+ °10 65 


+ 3cr 10 641 


+ 3o" 10 632 


+ 3cr 10 542 


+ a 10 5321 


+ ^975 


+ 2(7"974i 


+ 20" 9 732 


+ 2Cg651 


+ 6(T 9642 


+ 2a 96321 


+ 2cr 9543 


+ 2cr 95421 


+ a 8751 


+ 3cr g742 


+ °87321 


+ 3cr 8652 


+ 3cr 86 43 


+ 3cr 86421 


+ CT 85431 


+ a 7653 


+ a 76521 


+ cr 76431 





So e.g. we have: #( 5 3 3 i3) (11 6 4) = 1, #(5 3 3i 3 ) (10 64i) = 3, #(53313) ( 9741 ) = 2, and 

#(5 3 31 3 ) (9642) = 6. 

5. Linear relations between Stembridge's coefficients 

Combining (4), (12), and (16), we have in the above notation, associated with a 
fixed fx 

) ■ Qa#b \ (ai 1 ,&i k ) 

= ^2Q(c tl ,...,ci k ) ■ Qc#D^( Cil ,...,c ik ) = 2 n ^fiV<?A, 

A 

where the first two sums are over all sequences 1 < i\ < • • • < ik < n and k = 
0,1,... ,n. 

The equalities (18) imply linear relations between the e^'s and gx^s. Given 
a sequence of different positive integers K = (fci, . . . , ki), there is a permutation 
w = wk £ Si such that k w ^ > ■ ■ ■ > k w ^ > 0. Denote this last-mentioned strict 
partition by < K >. Then given strict partitions fx, A and a sequence K as above, 
we set 

(!9) e x ^ K := sgn(w K ) e\ <K> . 

T7„„™ /1 cA „„J „.„ -i-U„ -f„n„„.; — , „.."U. 
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Proposition 4. For a fixed partition fi and strict partition A with \fi\ = \X\, we 
have in the above notation associated with fi 

1 n 9\v = Xl e (aii.-,ai fc ), A#B^,(a il ,...,a ik ) 

-= V P x 

c '(c il ,Ci k ), C#D\(c il ,c ik ) 

where the sums are over all sequences 1 < i\ < ■ ■ ■ < it < n for which A#B \ 
(a^, ... ,di k ) (resp. Cj^D \ (c^, ... ,Ci k )) is a sequence of different integers, and 
k = 0, 1, . . . , n. 

For instance, for any strict partition A with |A| = 21, and for fx = (5 3 31 3 ) = 
(432|621), we get the equations: 

2 3 fi ( A (5 3 31 3 ) = e (654321) (0) ~ e (5) (64321) + e (4) (65321) ~ e (3) (65421) ~ e (54) (6321) 
+ e (53) (6421) ~~ e (43) (6521) + e (543) (621) 
= e (32) (7432) + e (732) (432)- 



6. The class i*(cr' x ) as a Z-linear combination of the <t m 's 

By reasoning similarly as in §4, one shows that for any proper morphism / : 
X — > G/P from a scheme X to a generalized flag variety G/P, and for any irre- 
ducible subscheme Y C X, f*([Y]) is a Z-linear combination of Schubert classes in 
H*(G/P;Z) with nonnegative coefficients. 

The next proposition will give i*(o~' x ) as an explicit Z-linear combination of the 
a M 's. Given a partition ^ C (n n ), we set /U* := (n — /U n , ... , n — The following 
duality property is a well-known result of Schubert calculus [F]: 

Lemma 2. The basis {o~ M } of the group H 2p (G; Z) and the basis {o~^* } 0/ the group 
are dual under the pairing (a, b) 1— > f G a ■ b of Poincare duality. 

We now state: 

Proposition 5. For a /ized s£nc£ partition Ac (n, n — 1, . . . ,1), we /iai>e 
(21) = fAv,^* o- M , 

| M | = |A|+n(n-l)/2 

where fx runs over partitions contained in (n n ) and #a v ,m* ^ s the Stembridge coeffi- 
cient described in Theorem (ii). 

Indeed, if i*(o~'x) = ^A/i^, with G Z (so that \fx\ = |A| + n(n — l)/2 ), 
then it follows from Lemma 2 that 




Using the projection formula for i, this is rewritten as 
(23) mxn = f o-' x • i*(<7 u *)- 
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In turn, using the description of i*(a^*) from Proposition 2, (23) is rewritten as 
(24) m x ^= <r'\- 9u»* <?'„)= 5252 e ^ 9v »* a 'r = '.m* 

because only r = (n, n — 1, . . . , 1) and v = A v give a nonzero contribution (note 
that for such r and z/, we have e\ v = 1). 

7. Relations between the degrees of the ordinary and projective repre- 
sentations of the symmetric groups 

For a partition /z, we set 

< 25 > : = n w Y 

where h(x) is the hook-length of \i at x = defined by h(x) =h(i,j) = fii+fxj — 

i — j + If = m then := ml is the degree of the irreducible representation 
of S m corresponding to \i. Equivalently, is the number of standard tableaux of 
shape fj,, obtained by labeling the squares of the diagram of fi with the numbers 
1,2,... , m. We refer to [F] for a detailed discussion of these facts. 

For a strict partition A, we set 

xes(\) v ' 

where S(X) is the shifted diagram associated with A [M, p. 255], and for each square 
x G S(X) the hook-length h(x) is defined to be the hook-length at x in the "double 
diagram" (Ai, A2, • • • |Ai — 1, A2 — 1, . . . ), containing S(X). If |A| = m, g x := ml g x 
is the number of shifted standard tableaux of shape <S(A), obtained by labeling 
the squares of S(X) with the numbers 1,2, .. . , m with strict increase along each 
row and down each column. The numbers g x also admit an interpretation as the 
degree of suitable projective representations of S m . We refer to [H-H] for a detailed 
discussion of these results. 

One has the following formulas, in terms of parts, for / [M, LI Example 1] and 
g x [M, III.8 Example 12]: 

[ ] ; " Ui>M + n-i)\ ' 
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Lemma 3. (i) Under the specialization e$ := jj , becomes / M . 
(ii) Under the specialization Qi := f t ■ , Q\ becomes g x . 

(For assertion (i), see [M, 1.3 Example 5]. Assertion (ii) stems from [DC-P, Propo- 
sition 6].) 

Given a partition fj,, we want to apply formulas (12) and (16), so we adopt the 
notation of §5. Also, we follow the notation of §6 associated with a sequence K. 
For such a sequence, we set 

(29) g K :=sgn(w K )g <K >. 

From Lemma 3, (12), and (16), we get 
Proposition 6. For a fixed partition \i, we have 

2™J^ _ y^^(a il ,...,a ifc ) -A#B\(a il ,a ifc ) 

(30) 

= ^j( c .i-A fc )jC#^(ci„-,Ci t ) ) 

where the sums are over all sequences 1 < %\ < ■ ■ ■ < ik < n for which A#B \ 
(a^, ... ,di k ) (resp. Cj^D \ (c^, ... ,Ci k )) is a sequence of different integers, and 
k = 0, 1, . . . , n. 

For instance, for [i = (5 3 31 3 ) = (432|621), we get the equations: 

2 3j(5 3 31 3 ) = _(654321) _ -(5) ^(64321) + -(4) ^(65321) _ -(3) ^(65421) _ -(54) ^(6321) 

+ ^ (53) g (m21) - g {43) g (6521) + ^ 543) gt 621 ') 
= 9 {32) ^ (7432) +g (732) ^ (432) . 
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